Content-based multiplexer device and multiplexing method

ABSTRACT

The multiplexer device comprises a multiplexing means for multiplexing a plurality types of media information and outputting a multiplexed bit stream, a priority deciding means for deciding priority corresponding to each of the media information, and a multiplexing controller for controlling multiplexing of each of the media information according to the multiplexing means based on the priority decided by the priority deciding means. In addition the priority in each information is multiplexed.

FIELD OF THE INVENTION

The present invention relates to a multiplexer device for multiplexingmultimedia data (media information) such as digitalized video signals,acoustical signals, and other data or the like. Multiplexed data isencoded to be transmitted or to be accumulated or for other purposes,and is applicable, as a concrete example of application, to a digitalbroadcasting system using satellites, ground waves, cables, or opticalfibers, video on demand (VOD), and a teleconference system or the like.

BACKGROUND OF THE INVENTION

FIG. 8 is a functional block diagram showing, as an example, aconventional type of multiplexing transmitting system described in thereport of the Telecommunications Council in the Ministry of Posts andTelecommunications, Consultation No. 74:

“Technological conditions in a satellite digital broadcasting system(which uses a 27 MHz band width), said satellite using a band width in arange from 12.2 to 12.75 GHz in “Technological conditions according to adigital broadcasting system”’(July, 1995).

In FIG. 8, designated at the reference numeral at 1 is a video encodingsection, at 2 a sound encoding section, at 3 a data encoding section, at4 a multiplexing section, at 5 a transmission scramble section, at 6 anerror correction encoding section, and at 7 modulating section.

Next description is made for operations of the multiplexing transmittingsystem based on the conventional technology.

In the video encoding section 1, sound encoding section 2, and dataencoding section 3, a video signal 11, an acoustic signal 12, andvarious types of data signal 13 are subjected to information-sourceencoding respectively.

The multiplexing section 4 multiplexes a video encoded bit stream 14, asound encoded bit stream 15, and data encoded bit stream 16 eachgenerated according to the information source encoding for each packetand generates a multiplexed bit stream 17.

The transmission scramble section 5 performs energy dispersal to themultiplexed bit stream 17. The error correction encoding section 6subjects a multiplexed bit stream 18 after the energy dispersal to errorcorrection encoding. The modulating section 7 subjects a multiplexed bitstream 19 after the error correction encoding to modulation suitable fora transmission line and generates a transmission signal 20 in aspecified frequency band.

Description is made for the multiplexing section 4 with reference toFIG. 9. In FIG. 9, the reference numeral 8 indicates a multiplexedadditional information generating section for generating informationindicating multiplexing and arrangement conditions or the like when eachof the encoded bit streams 14 to 16 are multiplexed. Multiplexedadditional information 21 such as PSI (Program Specific Information)specified in the ISO/IEC 13818-1 publication or SI (Service Information)specified in the European DVB (Digital Video Broadcasting) is generatedby the multiplexed additional information generating section 8.

The packet multiplexing section 9 multiplexes the video encoded bitstream 14, sound encoded bit stream 15, data encoded bit stream 16 andmultiplexed additional information 21 for each packet according to themultiplexed additional information 21 from the multiplexed additionalinformation generating section 8 and generates a multiplexed bit stream17.

The conventional type of multimedia multiplexing/transmitting system hasthe configuration as described above, which makes it difficult tomultiplex information by flexibly using any band according to thepriority in the information, for instance, by deciding the priority foreach type of video and speech or the like and deciding the priority toany object appearing in the video and speech or the like, namely, acomparatively more amount of information is allocated to the informationwith the higher priority to be multiplexed while a comparatively lessamount thereof is allocated to that with lower priority to bemultiplexed.

Also, in the side where the multiplexed bit stream is received, it isdifficult to edit and process the received information according to thepriority or for each object.

Further, since information is not hierarchically classified intoimportant information and not-important information to be transmitted,it is difficult to provide a function of stepped degradation; so calledthe function of graceful degradation in received information accordingto the quality of a transmission line.

SUMMARY OF THE INVENTION

It is an object of the present invention to enable multiplexing ofinformation flexibly and efficiently by multiplexing the informationaccording to the priority therein. It is another object of the presentinvention to realize multiplexer which has sufficient resistance to anerror of transmission line and also has a function of gracefuldegradation.

A multiplexer device according to the present invention comprises amultiplexing means for multiplexing a plurality types of mediainformation and outputting a multiplexed bit stream, a priority decidingmeans for deciding priority corresponding to each of the mediainformation, and a multiplexing controller for controlling multiplexingof each of the media information according to the priority decided bythe priority deciding means.

In the multiplexer device, the multiplexing means comprises a mediamultiplexer for multiplexing the plurality types of media informationand generating a media multiplexed bit stream for each program, and aprogram multiplexer for multiplexing a plurality of the mediamultiplexed bit streams outputted from this media multiplexer andgenerating a program multiplexed bit stream, the priority deciding meansdecides priority for each of the media information as well as for eachof the programs, and the multiplexing controller controls multiplexingof each of the media information by the media multiplexer according tothe priority for the media information and also controls multiplexing ofeach of the media multiplexed bit streams by the media multiplexeraccording to the priority for the program.

The multiplexer device comprises a channel coder for subjecting themultiplexed bit stream outputted from the multiplexing means to channelcoding according to the priority from the priority deciding means andtransmitting the subjected multiplexed bit stream.

In the multiplexer device, the priority deciding means decides priorityin each media information according to contents informationcorresponding to each of the media information, or decides priority ineach program according to contents information corresponding to each ofthe programs.

The multiplexer device comprises a priority correlation tableinformation generating means for generating both or one of informationindicating association and priority between media information or betweenprograms, and the multiplexing means multiplexes the prioritycorrelation table information.

In the multiplexer device, the priority correlation table informationcomprises packet identifying information for discretely identifying themedia information multiplexed for each packet, program identifyinginformation for identifying each program, group identifying informationindicating association between programs, and priority informationindicating the priority.

In the multiplexer device, further, the priority information comprisespriority information for each program and priority information for eachmedia information constituting each program.

A multiplexing/transmitting method according to the present inventioncomprises the steps of deciding priority corresponding to a pluralitytypes of media information, controlling multiplexing of each mediainformation according to the decided priority, and outputting amultiplexed bit stream.

Further scope of applicability of the present invention will becomeapparent from the detailed description given hereinafter. However, itshould be understood that the detailed description and specificexamples, while indicating preferred embodiments of the invention, aregiven by way of illustration only, since various changes andmodifications within the spirit and scope of the invention will becomeapparent to those skilled in the art from this detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will become more fully understood from thedetailed description given hereinbelow and the accompanying drawingswhich are given by way of illustration only, and thus are not limitativeof the present invention, and wherein:

FIG. 1 is a block diagram showing a multiplexing transmitter accordingto the embodiment of the present invention;

FIG. 2 is a block diagram showing an information-source encoding sectionin the multiplexing transmitter according to the embodiment of thepresent invention;

FIG. 3 is a block diagram showing a media multiplexing section in themultiplexing transmitter according to the embodiment of the presentinvention;

FIG. 4 is an explanatory view showing an example of priority correlationtable information in the multiplexing transmitter according to theembodiment of the present invention;

FIG. 5 is a block diagram showing a transmission scramble section in themultiplexing transmitter according to the embodiment of the presentinvention;

FIG. 6 is a block diagram showing an error correction encoding sectionin the multiplexing transmitter according to the embodiment of thepresent invention;

FIG. 7 is a flow chart showing each steps of the multiplexing methodaccording to the embodiment of the present invention;

FIG. 8 is a functional block diagram showing themultiplexing/transmitting system based on the conventional technology;and

FIG. 9 is a block diagram showing the multiplexing section in themultiplexing/transmitting system based on the conventional technology.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Description is made hereinafter for embodiments of the present invention

FIG. 1 is a block diagram showing configuration of the multiplexedtransmitter according to the present invention.

In the figure, designated at the reference numeral 31 is aninformation-source encoding section, at 32 a priority deciding sectionas a priority deciding means, at 33 a control section as a multiplexingcontroller, at 34 a media multipexer section as a program multiplexer,at 35 a multiplexed additional information generating section as apriority correlation table information generating means, at 36 a programmultiplexing section as a program multiplexer, at 37 a transmissionscramble section, at 38 an error correction encoding section, and at 39a modulating section. A channel coder comprises the transmissionscramble section 37, error correction encoding section 38, andmodulating section 39.

Next description is made for operations. The information-source encodingsection 31 comprises, as shown in FIG. 2 showing the configurationthereof, a plurality of video encoding sections 41, sound encodingsections 42, and data encoding sections 43.

Each of the encoding sections subjects a plurality of video signals 11,acoustic signals 12, and various types of data signal 13 each includedin inputted media signals 74 to information-source encoding according toinformation-source encoding characteristics (quantizing characteristics,an encoded bit rate, or the like) shown in a control signal 73 a fromthe control section 33, generates an encoded bit stream 75 comprisingvideo encoded bit streams 14, sound encoded bit streams 15, and dataencoded bit streams 16, and also generates a bit rate 82 comprising avideo encoded bit rate 82 v, a sound encoded bit rate 82 a, and a dataencoded bit rate 82 d each 20 corresponding to each media signal. Thebit rate 82 is sent to the control section 33.

The encoded bit streams 75 are sent to the media multiplexing section 34to be multiplexed.

The media multiplexing section 34 multiplexes, as shown in FIG. 3showing the configuration thereof, a plurality of media encoded bitstreams 75 for each packet, and generates a media multiplexed bit stream(a program bit stream) 76 for each program (broadcast program). Namely,a plurality of media (e.g. one video, two voices, one data) constitutingone unit of program are multiplexed as a single unit by a mediamultiplex-processing section 34 a. Multiplexed additional informationindicating which of the media information has been multiplexed is alsomultiplexed.

It should be noted that, as a method of selecting media informationcorresponding to a specified program from the encoded bit streams 75, ina case where relation between output from the information-sourceencoding section 31 and a program is fixed, the two may be connected toeach other by H/W, or may be variable by providing a switching functionin the previous stage. Further, S/W processing is also possible.

Herein, processing for multiplexing the encoded bit streams 75 by themedia multiplexing section 34 is controlled according to the prioritycorresponding to each media information indicated by a control signal 73b from the control section 33. This multiplexing control indicatespreferentially multiplexing of the information with higher priority bycontrolling ON/OFF of the multiplexing according to the priority foreach encoded bit stream 75 in a case where there is a limitation in abit rate per media multiplexed bit stream 76. (It should be noted thatmultiplexing or not multiplexing certain media information is supposedto control a transmission rate of the media information).

In the embodiment, the priority is decided as described below.

Contents information 71 for each media signal constituting the mediasignal 74 inputted in the information-source encoding section 31inputted in the priority deciding section 32.

The priority deciding section 32 decides priority of each media and thatfor each program (broadcast program) according to the contentsinformation 71, and generates an priority information 72.

For example, when a type of media (videos, sounds, various types of dataor the like), an information-source encoding system (non-layer encoding,layer encoding [base layer], layer encoding [enhancement layer] or thelike), contents of a broadcast program (movies, news, sports, or thelike), a mode of broadcast (ordinary broadcast/emergency broadcast, acurrent type of television/HDTV, monolingual/bilingual,monophonic/stereophonic or the like) or some other condition are givenaccording to the contents information 71, priority in each media as wellas priority in each program are decided-based on the previously decidedorder according to those contents. Higher priority in each program, forinstance, is supposed to be set in emergency news than in entertainmentbroadcast, while higher priority in each media is supposed to be set,among movie broadcast programs, in video information than in soundinformation.

It should be noted that, although a case of deciding priority from thecontents information 71 is described herein, the way of decidingpriority is not limited thereto, and, for instance, priority may bedecided by receiving a request from the side of receiving themultiplexing information.

Priority in each media and in each program is decided as described, issent to a control section 33 as priority information 72, and the controlsection 33 controls multiplexing of a media multiplexing section 34according to the control signal 73 b based on the sent information andprovides control so that preferential multiplexing is executed from themedia information with a higher priority in the same programs.

It should be noted that, in the media multiplexing section 34, timeinformation is cyclically added to a multiplexed packet when media aremultiplexed to ensure synchronous regeneration of multiplexed data inthe side of using the multiplexed one. As for a clock providing timeinformation, a different one may be used for each of media multiplexingin media multiplex-processing sections 34 a, . . . 34 n, respectively.

In a case where an encoded bit stream 75 includes a hierarchicallyencoded one in the information-source encoding section 31, encoded bitstreams for the corresponding base layer as well as enhancement layerare discretely subjected to media multiplexing, and discrete mediamultiplexed bit stream is generated. However, the bit streams aresubjected to media multiplexing respectively using a common clock toensure synchronous regeneration between the layers.

Further, the media multiplexing section 34 shown in FIG. 3 detects a bitrate 83 for each media multiplexed bit stream and sends the rate to thecontrol section 33.

Media multiplexed bit streams 76 for each program outputted from themedia multiplexing section 34 are multiplexed as a group obtained bymaking a single unit with a plurality of programs in the programmultiplexing section 36, and program multiplexed bit streams 78 for aplurality of groups (channels) are generated.

Herein, multiplex-processing of the media multiplexed bit stream 76 inthe program multiplexing section 36 is also controlled, as in the mediamultiplexing section 34, according to the priority corresponding to eachprogram indicated by a control signal 73 c from the control section 33.

This multiplexing control is to efficiently use a limited transmissionband (a bit rate) by selecting any program (an important program) to bemultiplexed from a plurality of media multiplexed bit streams 76, forinstance, in a case where a transmission band per program multiplexedbit stream is limited.

A method of deciding priority corresponding to each program is decidedby the priority deciding section 32 as described above.

The control section 33 controls multiplexing in the program multiplexingsection 36 according to a control signal 73 c based on the priority ineach program, and control is provided so that multiplexing ispreferentially executed from the higher priority program.

The program multiplexing section 36 also detects a bit rate 84 for eachprogram multiplexed bit stream and inputs the detected rate to thecontrol section 33.

As described above, control can be provided over multiplexing based onthe media information or the priority corresponding to each program, sothat multiplexing using a band efficiently can be carried out.

It should be noted that it is also possible to control multiplexing inconsideration of priority in each program when multiplexing is to becontrolled in the media multiplexing section 34, and reversely, it isalso possible to control multiplexing in consideration of priority ineach media when multiplexing is to be controlled in the programmultiplexing section 36.

Herein, description is made for the multiplexed additional informationgenerating section 35. The section is provided for multiplexinginformation indicating priority therein for using the multiplexedinformation when multiplexing is controlled according to the prioritytherein, so that processing such as output and edit/process or the likecan be executed for each priority when the multiplexed information is tobe used.

The multiplexed additional information generating section 35 generates,like the multiplexed additional information generating section 8 in theconventional type of multiplexing/transmitting system shown in FIG. 8,multiplexed additional information 77 indicating conditions ofmultiplexing/arrangement of each of encoded bit stream or the like.

However, the multiplexed additional information generating section 35 isdifferent from the conventional type of multiplexed additionalinformation generating section 8 in a point that the section 35generates priority correlation table information described below.

The priority correlation table information comprises, as shown in FIG.4, packet, identifying information 35 a for identifying each encoded bitstream 75 multiplexed for each packet in the information-source encodingsection 31, program identifying information 35 b for identifying eachmedia multiplexed bit stream 76 obtained by that encoded bit streams 75are multiplexed for each program in the media multiplexing section 34 tobe outputted, group identifying information 35 c for indicatingassociation between the media multiplexed bit streams 76 multiplexed foreach group in the program multiplexing section 36, and priorityinformation 35 d indicating priority of each packet and program.

The priority information 35 d further comprises information for priorityin a program, information for priority among programs, information forpriority in a group, and information for priority among groups. In thisexample, decision of the priority is executed according to the priorityinformation 72 and control information 73.

The information for priority in a program indicates priority of anencoded bit stream for various types of media which are elementsconstituting a program and is obtained by referring to the priorityinformation 72 decided in the priority deciding section 32.

The information for priority among programs indicates priority for eachmedia multiplexed bit stream included in the same group made in theprogram multiplexing section 36. It is assumed that time informationobtained from the common clock is added to a media multiplexed bitstream having the same group information.

With this feature, synchronous regeneration of multiplexed data isinsured between the media multiplexed bits each having the same groupinformation. It is also assumed that media multiplexed bit streams eachhaving the same group information are all transmitted over the sametransmission channel while one or a plurality of groups are transmittedfor each transmission channel.

For example, media multiplexed bit streams obtained by multiplexinglayered data which is relative to each other have the same groupinformation, and the media multiplexed bit stream obtained bymultiplexing data for standard quality has the highest priority in thegroup, while the media multiplexed bit stream obtained by multiplexingdata for high quality has the lower priority than the above stream inthe group.

The information for priority in a group indicates priority in encodedbit streams for various types of media which are elements in the samegroup and is obtained by referring to the priority information 72 andthe control information 73.

The information for priority among groups indicates priority of eachgroup and is set manually or by referring to the priority information 72and control information 73. For instance, the highest priority amonggroups is allocated to the group including a multiplexed packet with thehighest priority in a group first, and the second highest priority amonggroups is allocated to a multiplexed packet with the second highestpriority and for forth, and in a case where maximum values of eachpriority in a group are identical, priority among groups is decided ineach group by comparing the second highest priority in a group to eachother.

It should be noted that it is assumed in the embodiment that “1” isallocated to the highest priority and a numerical values becomes largerin accordance with lower priority. Allocation of priority may be a wayof relative allocation or a way of absolute allocation.

Further, priority information for priority correlation table comprisesthe four types of priority information as described above in theembodiment, however, the priority information 72 may be used as it is toset priority information for each multiplexed packet. Some of thepriority information may be omitted. For example, in a case where any ofthe program multiplexed bit streams 78 has the same group information,both of or one of the priority information among groups and groupinformation may be omitted.

The priority correlation table information generated as described aboveis sent to the program multiplexing section 36 as multiplexed additionalinformation 77 to be added (multiplexed) to each channel. With thisoperation, when multiplexed information is to be used, processing suchas output, edit/process or the like can be executed for each priorityusing the priority correlation table information.

The embodiment relates to a multiplexed transmitter for transmittingmultiplexed information, and description is made hereinafter foroperations of a section corresponding to the channel coding means.

As described above, program multiplexed bit streams 78 for a pluralityof channels outputted from the program multiplexing section 36 are sentto the transmission scramble section 37. FIG. 5 is a block diagramshowing an example of configuration of the transmission scramble section37. The transmission scramble section 37 receives the programmultiplexed bit streams 78 for a plurality of channels, subjects eachprogram multiplexed bit stream 78 to processing by each of transmissionscramble-processing sections 37 a, . . . 37 n, and outputs the programmultiplexed bit streams 79 for a plurality of channels subjected toenergy dispersal. Each transmission scramble characteristics iscontrolled according to a control signal 73 d from the control section33.

The program multiplexed bit streams 79 are sent to the error correctionencoding section 38. FIG. 6 is a block diagram showing an example ofconfiguration of the error correction encoding section 38. The errorcorrection encoding section 38 receives the program multiplexed bitstreams 79 for a plurality of channels subjected to energy dispersal,subjects each program multiplexed bit stream to error correctionencoding by each of error correction encoding-processing sections 38 a,..., 38 n, and outputs the program multiplexed bit streams 80 for aplurality of channels subjected to error correction encoding. Each errorcorrection encoding characteristics is controlled according to a controlsignal 73 e therefrom.

A modulating section 39 modulates the program multiplexed bit streams 80for a plurality of channels subjected to error correction encoding,outputs a transmission signal 81 for one or a plurality of channels, andthe signal(s) is transmitted.

Herein, the modulation is controlled according to a control signal 73 f.In the embodiment, OFDM (Orthogonal Frequency Division Multiplexing) isused for a modulating system. As for a modulating system for eachcarrier, according to a control signal 73, 64 QAM (Quadrature AmplitudeModulation) which is high in frequency utilized efficiency is used for alow-priority program multiplexed bit stream, and QPSK (Quadrature PhaseShift Keying) which is high in error resistance is used for ahigh-priority program multiplexed bit stream.

The modulating section 39 generates, in a case where the OFDM(Orthogonal Frequency Division Multiplexing) is employed as themodulating system therein, a program multiplexed bit stream 78corresponding to each carrier.

It should be noted that, as described above, the control section 33generates control signals 73 a to 73 f for appropriately controllinginformation-source encoding characteristics (quantizing characteristics,encoded bit rate or the like), media multiplexing, generation ofmultiplexed additional information, program multiplexing, a transmissionscramble system, an error correction encoding system, and a modulatingsystem according to a bit rate 82 for information-source encoding fromthe priority information 72 as well as from the information-sourceencoding section 31, a bit rate 83 for each media multiplexed bit streamfrom the media multiplexing section 34, and a bit rate 84 for eachprogram multiplexed bit stream from the program multiplexing section 36.

In the embodiment, the media multiplexing, program multiplexing,transmission scramble system, error correction encoding system, and themodulating system are appropriately controlled according to the controlsignals 73 a to 73 f, so that it is required to transmit the contents ofcontrol to a receiving side. Accordingly, those controlled contents aremodulated as the highest priority signals to be transmitted on a carrieror a transmission channel which is strongest in error resistance.

In the embodiment, by making encoded bit streams 75 for a plurality ofmedia to encoded bit streams each for a different object respectivelyand setting that priority information 72 is indicated as priority ofeach object, hierarchically classified information according to eachobject can efficiently be multiplexed for transmission and hassufficient resistance to an error of transmitting channel. In addition,information is multiplexed for each object, so that the receivedinformation can easily be edited or processed for each object in theside of receiver.

Further, in the embodiment, error resistance is strengthened toimportant information to be transmitted, so that it is possible torealize phased degradation so-called graceful degradation in receivedinformation according to quality of a transmission line.

Next description is made for operations in accordance with the flowchart of FIG. 7.

In this step, a priority corresponding to a plurality types of mediainformation is decided (step S1).

In second step, a multiplexing of each media information is controlledaccording to the decided priority (step S2).

In the third step, a multiplexed bit stream is outputted (step S3).

It should be noted that each section in the above embodiment can berealized by means of both a hardware circuit and a software circuit.

As described above, with the present invention, media information orprogram information is controlled for multiplexing according to thepriority therein, which makes it possible to efficiently multiplexinformation according to the priority.

It is also possible to efficiently transmit information multiplexedcorresponding to the priority by means of a channel coder.

In addition, priority information is multiplexed, which makes itpossible to edit and process the multiplexed information for eachpriority in the side of user.

This application is based on Japanese patent application No. HEI8-228317 filed in the Japanese Patent Office on Aug. 29, 1996, theentire contents of which are hereby incorporated by reference.

Although the invention has been described with respect to a specificembodiment for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art which fairly fall within the basic teaching hereinset forth.

What is claimed is:
 1. A multiplexer device comprising: a multiplexerfor multiplexing a plurality of media information and input prioritytable information, wherein the priority table information includespacket identifying information for identifying each packet in amultiplexed bit stream, program identifying information for identifyingeach program composed of the media information, group identifyinginformation indicating an association among programs, and priorityinformation indicating priority.
 2. The multiplexer device according toclaim 1, wherein the priority information includes priority of eachprogram and priority of each media information within each program.
 3. Amethod of multiplexing media information, comprising: multiplexing aplurality of media information on a packet basis, wherein, whenmultiplexing packets of the plurality of media information, packetidentifying information for identifying packets corresponding to eachmedia information in a multiplexed bit stream, information indicatingcorrespondence between media information, and information indicatingpriority of the media information are multiplexed.
 4. The method ofmultiplexing media information according to claim 3, wherein theinformation indicating priority of the media information is providedhierarchically in correspondence to the information indicatingcorrespondence among the media information.
 5. The method ofmultiplexing media information according to claim 3, wherein theinformation indicating priority of the media information includespriority information indicating priority of a program and priorityinformation indicating priority of each medium which forms the program.6. The method of multiplexing media information according to claim 3,wherein the media information is encoded objects and the informationindicating correspondence among said media information is informationindicating a relation between objects.
 7. The method of multiplexingmedia information according to claim 3, wherein the informationindicating priority of the media information includes a priority valuethat represents different priority values represent different prioritylevels.